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Error Detection and Correction: Self and Automatic 


• Human beings make and usually detect errors routinely. The same mental processes 
that allow humans to cope with novel problems can also lead to error. Bill Rouse has 
argued that errors are not inherently bad but their consequences may be. He proposes 
the development of “error-tolerant” systems that detect errors and take steps to prevent 
the consequences of the error from occurring. Research should be done on self and 
automatic detection of random and unanticipated errors. For self detection, displays should 
be developed that make the consequences of errors immediately apparent. For example, 
electronic map displays graphically show the consequences of horizontal flight plan entry errors. 
Vertical profile displays should be developed to make apparent vertical flight planning errors. 
Other concepts such as “energy circles” could also help the crew detect gross flight planning 
errors. For automatic detection, systems should be developed that can track pilot activity, 
infer pilot intent and inform the crew of potential errors before their consequences are 
realized. Systems that perform a reasonableness check on flight plan modifications by checking 
route length and magnitude of course changes are simple examples. Another example would 
be a system that checked the aircraft’s planned altitude against a data base of world terrain 
elevations. 


From: Flight Deck Automation: Promises and Realities 
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Error Detection & Correction: Self and Automatic 

• Humans make and usually detect errors routinely. 

• The same mental processes that allow humans to cope with novel problems 

can also lead to error. 

• Errors are not inherently bad but their consequences may be. 

• "Error-Tolerant" Systems should be developed that can track pilot activity, 

infer pilot intent and inform the crew of potential errors. 

From: Flight Deck Automation: Promises and Realities 


Research Goal 

• To design systems that can infer the crew's current plan, form 

expectations about future crew actions and warn the crew of possible 
errors. 


Approach: 

• Base the system on script based Al programs that 
understand human actions in stories. 

• Develop a hierarchical script based program to detect 
procedural errors in data form our B-727 simulator. 

• Incorporate the program concepts into a "SMART 
CHECKLIST" for the Advanced Cockpit Flight Simulator". 

• Support Related Grant and Contract Research. 
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OBJECTIVE 


APPROACH 


• AVIONIC SYSTEMS THAT "UNDERSTAND" 
THE ACTIONS OF CREW AND CAN 
INFORM CREW OF POSSIBLE ERRORS 


• SCRIPT BASED MODEL 
•TRACK CREW ACTIONS 

• DETECT ERRORS IN B-727 SIMULATOR 

• DETERMINE ERROR CONSEQUENCES 

• REAL-TIME FEEDBACK 


• SMART CHECKLISTS FOR THE ACFS 

AIRCRAFT CREW SCRIPT AIRCRAFT FLIGHT 

STATE ACTIONS MOOR. MOOR. PLAN 



SYSTEM ARCHITECTURE 


SCRIPT OF CREW ACTIVITIES 


Status 

• B-727 flights analysed with Version 1 of the script based 

activity tracking program. 

• Difficulty in dealing with actions from procedures done in 

an unexpected order. 

• Version 2 of the script based activity tracking program 

"explains” observed actions by linking them to expected 
actions in the procedure script. 

• Gathered data on procedure execution in two full mission experiments 

in our 727 simulator. 

Plans 

• Analyze 727 data from the "ATC FLOW" and "PNPS" Experiments. 

• Compare program to pilot understanding of crew activity. 

• Compare program to "OFMspert” developed at Georgia Tech. 

• Develop and test Smart Checklists in the ACFS. 


ORIGINAL PAGE IS 
OF POOR QUALITY 
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Two Problems with Conventional Checklists 


• External Memory. 

• Task Automization. 

Smart Checklists Designs 

• Designs are based on the Script Based Procedure Tree Architecture. 

• Phase of Flight and Procedure Selection will be done Manually. 

• Designs differ in the Level of Automation of procedural tasks. 

• Designs differ in the Level of involvement of the crew in the 
execution and monitoring of procedural tasks. 


Normal Checklists 


Preflight 

Before Engine Start 
After Engine Start 

\ 

ACFS 

Checklists J 
w — ' 

Normal j 

Checklists) 

r - -V 

Before Takeoff 
After Takeoff 
Descent & Approach 

f o , ^ 

Before 

Landing (1) J 

\ 

Before 

Landing (2) J 

Before Landinq 

After 

Landing J 

After Landing 
Shutdown 
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Before Landing - Page 2 of 2 


Seat Belt Light 
No Smoking Light 

Spoilers 

Landing Gear 
Flaps 

Landing Clearance 


Armed 


Down 


Down 


Received 


ACFS 

Checklists j 
\ 

Normal 
Checklists > 

\ 

Before 

Landing (1)^ 

Before | 
Landing (2 )\ 

\ 

After 

Landing , 


Engine Overheat 


Engine Bleed Air Switch Off Normal 


.Retard 


Thrust Lever 

Retard slowly until ENG OVHT 
light extinguishes. 


Is ENG OVH light still illuminated? | YES| | NO | 
• Engine Failure / Shutdown 

Checklist Accomplish 


Is wing anti-ice required? [YES | | NO | 

• One Pack Control Selector Off 

• Isolation Switch (Affected Side) On 

Return to OFF when anti-ice is 

no longer required. 


Non-Normal 
Checklists J 
N 

ACAWS 
Checklists > 

' > 

Engine 
Overheat > 
s 

Before 
Landing > 


End of Engine Overheat Checklist *** 
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Engine Overheat - Page 1 


Engine Bleed Air Switch Off 

Thrust Lever Retard 

Retard slowly until ENG OVHT 
light extingui shes. 

Is ENG OVH light illuminated? YES no 

• Engine Failure / Shutdown 

Checklist Accomplish 


Is Wing Anti-Ice Required? |YES| | NU | 

• One Pack Control Selector Off 

• Isolation Switch (Affected Side) On 

Return to OFF when anti-ice is 

no longer required. 

*** End of Engine Overheat Checklist *** 


Normal 
Checklists j 

n 

Non-Normal 
Checklists v 

N 

ACAWS 

Checklists . 

" < 

Engine 
Overheat . 

r— 

s 

Before 

Landing 


Checklist Features - Experimental Conditions 


• A Passive Electronic Checklist -> External Memory of completed steps. 

• A Monitored Electronic Checklist -> Machine Monitoring of crew actions 

• An Automatic Checklist Control -> Lower Workload 

• An Automatic Execution Checklist -> Still Lower Workload 
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Expected Results of Research 

• Reduce consequences of pilot error. 

• A model of the pilot for the avionic system. 

• Avionic systems that "understand" pilot intent. 

• Avionic systems that knows the current context. 

• A framework for electronic checklists. 

• Data on human error. 


Related Grants and Contracts 

• "Bayesian Temporal Reasoning" 

- Curry, Cooper & Horvitz at Search Technology Inc. 

• "Operator Function Modeling & OFMspert" 

- Mitchell at Georgia Institute of Technology 

• "Expert Flight Systems Monitor" 

- Frogner, Jain & Phatac at Expert Ease Systems Inc. 

• "Distributed Cognition in Aviation" 

- Norman & Hutchins at University of California, San Diego 

• "Human Factors of Flight Deck Checklists" 

- Degani at University of Miami. 
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Two Dimensions of Automation: Control & Monitoring 

Monitor Functions 


manual 


auto 


EXPLORE 

THIS 

CORNER 



manual 


The objective of this research is to develop the technology necessary for the design of error tolerant 
cockpits. A key feature of error-tolerant systems is that they incorporate a model of pilot behavior. 
The system uses this model to track pilot actions, infer pilot intent, detect unexpected actions, and 
alert the crew to potential errors. In some sense, the goal is to develop an "electronic check pilot" 
that can intelligently monitor pilot activities. 

We are pursuing a number of alternative ways to track operator activity and infer operator intent. We 
are investigating techniques based on 1) a rule based script of flight phases and procedural actions, 2) 
operator function models, and 3) Bayesian temporal reasoning. The first version of the script based 
program was tested against protocol data from four 727 simulator flights. The program could detect 
procedural errors but its ability to account for pilot actions from procedures done out of the normal 
sequence was inadequate. A capability to explain unexpected actions by linking them to procedures 
that are nominally done or unstarted is being added to the program to remedy this problem. Under a 
grant to Georgia Tech, an intent inferencing system based on ah operator function model was developed 
and tested on data from a satellite communications system with good results. Under a contract to 
Search Technology, a prototype for an intent inferencing system based on Baysian reasoning was 
developed. We plan to compare these methods against data from our 727 simulator. We also plan to 
initiate an empirical study designed to better understand how check pilots detect procedural errors 
and infer pilot intent. 

The technology developed for the "Procedural Error Monitor" will be used to develop an interactive 
cockpit display to aid pilots in executing procedures. Modes of checklist operation will include both 
passively monitoring pilot execution of procedures and automatically executing procedures. Under a 
related SBIR contract, we will develop and test a procedure execution aid that can compose procedures 
that are appropriate for the current flight situation and equipment configuration. 

Everett A. Palmer 
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